Optimal Model Placement and Online Model Splitting for Device-Edge Co-Inference

Device-edge co-inference opens up new possibilities for resource-constrained wireless devices (WDs) to execute deep neural network (DNN)-based applications with heavy computation workloads. In particular, the WD executes first few layers of DNN and sends intermediate features edge server that processes remaining DNN. By adapting model splitting decision, there exists a tradeoff between local cost communication overhead. practice, is re-trained updated periodically at server. Once parameters are regenerated, part must be placed facilitate on-device inference. this paper, we study joint optimization placement online decisions minimize energy-and-time device-edge in presence channel fading. The problem challenging because strongly coupled, while involving two different time scales. We tackle by formulating an optimal stopping problem, where finite horizon determined decision. addition deriving rule based on backward induction, further investigate simple one-stage look-ahead rule, which able obtain analytical expressions analysis useful us efficiently optimize decision larger scale. closed-form solution fully-connected multilayer perceptron equal neurons. Simulation results validate superior performance various structures.